Filtering Rows¶
# import pandas
import pandas as pd
# read movie data
movies = pd.read_csv("http://bit.ly/imdbratings")
# examine first few rows
movies.head()
| star_rating | title | content_rating | genre | duration | actors_list | |
|---|---|---|---|---|---|---|
| 0 | 9.3 | The Shawshank Redemption | R | Crime | 142 | [u'Tim Robbins', u'Morgan Freeman', u'Bob Gunt... |
| 1 | 9.2 | The Godfather | R | Crime | 175 | [u'Marlon Brando', u'Al Pacino', u'James Caan'] |
| 2 | 9.1 | The Godfather: Part II | R | Crime | 200 | [u'Al Pacino', u'Robert De Niro', u'Robert Duv... |
| 3 | 9.0 | The Dark Knight | PG-13 | Action | 152 | [u'Christian Bale', u'Heath Ledger', u'Aaron E... |
| 4 | 8.9 | Pulp Fiction | R | Crime | 154 | [u'John Travolta', u'Uma Thurman', u'Samuel L.... |
Filtering Movies with for Loop¶
booleans = []
for length in movies.duration:
if length >= 200:
booleans.append(True)
else:
booleans.append(False)
# Check length of booleans
len(booleans)
979
# Inspect booleans elements
booleans[0:5]
[False, False, True, False, False]
# create a pandas series
is_long = pd.Series(booleans)
# Inspect few values
is_long.head()
0 False
1 False
2 True
3 False
4 False
dtype: bool
# show dataframe all columns in duration 200 minutes
movies[is_long]
| star_rating | title | content_rating | genre | duration | actors_list | |
|---|---|---|---|---|---|---|
| 2 | 9.1 | The Godfather: Part II | R | Crime | 200 | [u'Al Pacino', u'Robert De Niro', u'Robert Duv... |
| 7 | 8.9 | The Lord of the Rings: The Return of the King | PG-13 | Adventure | 201 | [u'Elijah Wood', u'Viggo Mortensen', u'Ian McK... |
| 17 | 8.7 | Seven Samurai | UNRATED | Drama | 207 | [u'Toshir\xf4 Mifune', u'Takashi Shimura', u'K... |
| 78 | 8.4 | Once Upon a Time in America | R | Crime | 229 | [u'Robert De Niro', u'James Woods', u'Elizabet... |
| 85 | 8.4 | Lawrence of Arabia | PG | Adventure | 216 | [u"Peter O'Toole", u'Alec Guinness', u'Anthony... |
| 142 | 8.3 | Lagaan: Once Upon a Time in India | PG | Adventure | 224 | [u'Aamir Khan', u'Gracy Singh', u'Rachel Shell... |
| 157 | 8.2 | Gone with the Wind | G | Drama | 238 | [u'Clark Gable', u'Vivien Leigh', u'Thomas Mit... |
| 204 | 8.1 | Ben-Hur | G | Adventure | 212 | [u'Charlton Heston', u'Jack Hawkins', u'Stephe... |
| 445 | 7.9 | The Ten Commandments | APPROVED | Adventure | 220 | [u'Charlton Heston', u'Yul Brynner', u'Anne Ba... |
| 476 | 7.8 | Hamlet | PG-13 | Drama | 242 | [u'Kenneth Branagh', u'Julie Christie', u'Dere... |
| 630 | 7.7 | Malcolm X | PG-13 | Biography | 202 | [u'Denzel Washington', u'Angela Bassett', u'De... |
| 767 | 7.6 | It's a Mad, Mad, Mad, Mad World | APPROVED | Action | 205 | [u'Spencer Tracy', u'Milton Berle', u'Ethel Me... |
Filtering by Condition¶
# filtering by conditions
is_long = movies.duration >= 200
is_long.head()
0 False
1 False
2 True
3 False
4 False
Name: duration, dtype: bool
# show the rows duration >200
movies[is_long]
| star_rating | title | content_rating | genre | duration | actors_list | |
|---|---|---|---|---|---|---|
| 2 | 9.1 | The Godfather: Part II | R | Crime | 200 | [u'Al Pacino', u'Robert De Niro', u'Robert Duv... |
| 7 | 8.9 | The Lord of the Rings: The Return of the King | PG-13 | Adventure | 201 | [u'Elijah Wood', u'Viggo Mortensen', u'Ian McK... |
| 17 | 8.7 | Seven Samurai | UNRATED | Drama | 207 | [u'Toshir\xf4 Mifune', u'Takashi Shimura', u'K... |
| 78 | 8.4 | Once Upon a Time in America | R | Crime | 229 | [u'Robert De Niro', u'James Woods', u'Elizabet... |
| 85 | 8.4 | Lawrence of Arabia | PG | Adventure | 216 | [u"Peter O'Toole", u'Alec Guinness', u'Anthony... |
| 142 | 8.3 | Lagaan: Once Upon a Time in India | PG | Adventure | 224 | [u'Aamir Khan', u'Gracy Singh', u'Rachel Shell... |
| 157 | 8.2 | Gone with the Wind | G | Drama | 238 | [u'Clark Gable', u'Vivien Leigh', u'Thomas Mit... |
| 204 | 8.1 | Ben-Hur | G | Adventure | 212 | [u'Charlton Heston', u'Jack Hawkins', u'Stephe... |
| 445 | 7.9 | The Ten Commandments | APPROVED | Adventure | 220 | [u'Charlton Heston', u'Yul Brynner', u'Anne Ba... |
| 476 | 7.8 | Hamlet | PG-13 | Drama | 242 | [u'Kenneth Branagh', u'Julie Christie', u'Dere... |
| 630 | 7.7 | Malcolm X | PG-13 | Biography | 202 | [u'Denzel Washington', u'Angela Bassett', u'De... |
| 767 | 7.6 | It's a Mad, Mad, Mad, Mad World | APPROVED | Action | 205 | [u'Spencer Tracy', u'Milton Berle', u'Ethel Me... |
Filtering in DataFrame¶
# filtering by columns
movies[movies.duration >= 200]
| star_rating | title | content_rating | genre | duration | actors_list | |
|---|---|---|---|---|---|---|
| 2 | 9.1 | The Godfather: Part II | R | Crime | 200 | [u'Al Pacino', u'Robert De Niro', u'Robert Duv... |
| 7 | 8.9 | The Lord of the Rings: The Return of the King | PG-13 | Adventure | 201 | [u'Elijah Wood', u'Viggo Mortensen', u'Ian McK... |
| 17 | 8.7 | Seven Samurai | UNRATED | Drama | 207 | [u'Toshir\xf4 Mifune', u'Takashi Shimura', u'K... |
| 78 | 8.4 | Once Upon a Time in America | R | Crime | 229 | [u'Robert De Niro', u'James Woods', u'Elizabet... |
| 85 | 8.4 | Lawrence of Arabia | PG | Adventure | 216 | [u"Peter O'Toole", u'Alec Guinness', u'Anthony... |
| 142 | 8.3 | Lagaan: Once Upon a Time in India | PG | Adventure | 224 | [u'Aamir Khan', u'Gracy Singh', u'Rachel Shell... |
| 157 | 8.2 | Gone with the Wind | G | Drama | 238 | [u'Clark Gable', u'Vivien Leigh', u'Thomas Mit... |
| 204 | 8.1 | Ben-Hur | G | Adventure | 212 | [u'Charlton Heston', u'Jack Hawkins', u'Stephe... |
| 445 | 7.9 | The Ten Commandments | APPROVED | Adventure | 220 | [u'Charlton Heston', u'Yul Brynner', u'Anne Ba... |
| 476 | 7.8 | Hamlet | PG-13 | Drama | 242 | [u'Kenneth Branagh', u'Julie Christie', u'Dere... |
| 630 | 7.7 | Malcolm X | PG-13 | Biography | 202 | [u'Denzel Washington', u'Angela Bassett', u'De... |
| 767 | 7.6 | It's a Mad, Mad, Mad, Mad World | APPROVED | Action | 205 | [u'Spencer Tracy', u'Milton Berle', u'Ethel Me... |
# select only genre
movies[movies.duration >= 200].genre
2 Crime
7 Adventure
17 Drama
78 Crime
85 Adventure
142 Adventure
157 Drama
204 Adventure
445 Adventure
476 Drama
630 Biography
767 Action
Name: genre, dtype: object
# same as above
movies[movies.duration >= 200]['genre']
2 Crime
7 Adventure
17 Drama
78 Crime
85 Adventure
142 Adventure
157 Drama
204 Adventure
445 Adventure
476 Drama
630 Biography
767 Action
Name: genre, dtype: object
# select columns by label
movies.loc[movies.duration >= 200, 'genre']
2 Crime
7 Adventure
17 Drama
78 Crime
85 Adventure
142 Adventure
157 Drama
204 Adventure
445 Adventure
476 Drama
630 Biography
767 Action
Name: genre, dtype: object
Multiple Filtering Criteria¶
True and True == True
True and False == False
True or True == True
True or False == True
False or False == False
True and True
True
True and False
False
# multiple criteria
movies[(movies.duration >= 200) & (movies.genre == 'Drama')]
| star_rating | title | content_rating | genre | duration | actors_list | |
|---|---|---|---|---|---|---|
| 17 | 8.7 | Seven Samurai | UNRATED | Drama | 207 | [u'Toshir\xf4 Mifune', u'Takashi Shimura', u'K... |
| 157 | 8.2 | Gone with the Wind | G | Drama | 238 | [u'Clark Gable', u'Vivien Leigh', u'Thomas Mit... |
| 476 | 7.8 | Hamlet | PG-13 | Drama | 242 | [u'Kenneth Branagh', u'Julie Christie', u'Dere... |
# multiple criteria
movies[(movies.duration >= 200) | (movies.genre == 'Drama')]
| star_rating | title | content_rating | genre | duration | actors_list | |
|---|---|---|---|---|---|---|
| 2 | 9.1 | The Godfather: Part II | R | Crime | 200 | [u'Al Pacino', u'Robert De Niro', u'Robert Duv... |
| 5 | 8.9 | 12 Angry Men | NOT RATED | Drama | 96 | [u'Henry Fonda', u'Lee J. Cobb', u'Martin Bals... |
| 7 | 8.9 | The Lord of the Rings: The Return of the King | PG-13 | Adventure | 201 | [u'Elijah Wood', u'Viggo Mortensen', u'Ian McK... |
| 9 | 8.9 | Fight Club | R | Drama | 139 | [u'Brad Pitt', u'Edward Norton', u'Helena Bonh... |
| 13 | 8.8 | Forrest Gump | PG-13 | Drama | 142 | [u'Tom Hanks', u'Robin Wright', u'Gary Sinise'] |
| ... | ... | ... | ... | ... | ... | ... |
| 958 | 7.4 | My Sister's Keeper | PG-13 | Drama | 109 | [u'Cameron Diaz', u'Abigail Breslin', u'Alec B... |
| 968 | 7.4 | The English Patient | R | Drama | 162 | [u'Ralph Fiennes', u'Juliette Binoche', u'Will... |
| 970 | 7.4 | Wonder Boys | R | Drama | 107 | [u'Michael Douglas', u'Tobey Maguire', u'Franc... |
| 972 | 7.4 | Blue Valentine | NC-17 | Drama | 112 | [u'Ryan Gosling', u'Michelle Williams', u'John... |
| 973 | 7.4 | The Cider House Rules | PG-13 | Drama | 126 | [u'Tobey Maguire', u'Charlize Theron', u'Micha... |
287 rows × 6 columns
# multiple or conditions
movies[(movies.genre == "Crime") | (movies.genre == 'Drama') | (movies.genre == "Action")]
| star_rating | title | content_rating | genre | duration | actors_list | |
|---|---|---|---|---|---|---|
| 0 | 9.3 | The Shawshank Redemption | R | Crime | 142 | [u'Tim Robbins', u'Morgan Freeman', u'Bob Gunt... |
| 1 | 9.2 | The Godfather | R | Crime | 175 | [u'Marlon Brando', u'Al Pacino', u'James Caan'] |
| 2 | 9.1 | The Godfather: Part II | R | Crime | 200 | [u'Al Pacino', u'Robert De Niro', u'Robert Duv... |
| 3 | 9.0 | The Dark Knight | PG-13 | Action | 152 | [u'Christian Bale', u'Heath Ledger', u'Aaron E... |
| 4 | 8.9 | Pulp Fiction | R | Crime | 154 | [u'John Travolta', u'Uma Thurman', u'Samuel L.... |
| ... | ... | ... | ... | ... | ... | ... |
| 970 | 7.4 | Wonder Boys | R | Drama | 107 | [u'Michael Douglas', u'Tobey Maguire', u'Franc... |
| 972 | 7.4 | Blue Valentine | NC-17 | Drama | 112 | [u'Ryan Gosling', u'Michelle Williams', u'John... |
| 973 | 7.4 | The Cider House Rules | PG-13 | Drama | 126 | [u'Tobey Maguire', u'Charlize Theron', u'Micha... |
| 976 | 7.4 | Master and Commander: The Far Side of the World | PG-13 | Action | 138 | [u'Russell Crowe', u'Paul Bettany', u'Billy Bo... |
| 978 | 7.4 | Wall Street | R | Crime | 126 | [u'Charlie Sheen', u'Michael Douglas', u'Tamar... |
538 rows × 6 columns
# multiple or using isin() method
movies.genre.isin(["Drama", "Action", "Crime"])
0 True
1 True
2 True
3 True
4 True
...
974 False
975 False
976 True
977 False
978 True
Name: genre, Length: 979, dtype: bool
# pass the series in DataFrame
movies[movies.genre.isin(["Drama", "Action", "Crime"])]
| star_rating | title | content_rating | genre | duration | actors_list | |
|---|---|---|---|---|---|---|
| 0 | 9.3 | The Shawshank Redemption | R | Crime | 142 | [u'Tim Robbins', u'Morgan Freeman', u'Bob Gunt... |
| 1 | 9.2 | The Godfather | R | Crime | 175 | [u'Marlon Brando', u'Al Pacino', u'James Caan'] |
| 2 | 9.1 | The Godfather: Part II | R | Crime | 200 | [u'Al Pacino', u'Robert De Niro', u'Robert Duv... |
| 3 | 9.0 | The Dark Knight | PG-13 | Action | 152 | [u'Christian Bale', u'Heath Ledger', u'Aaron E... |
| 4 | 8.9 | Pulp Fiction | R | Crime | 154 | [u'John Travolta', u'Uma Thurman', u'Samuel L.... |
| ... | ... | ... | ... | ... | ... | ... |
| 970 | 7.4 | Wonder Boys | R | Drama | 107 | [u'Michael Douglas', u'Tobey Maguire', u'Franc... |
| 972 | 7.4 | Blue Valentine | NC-17 | Drama | 112 | [u'Ryan Gosling', u'Michelle Williams', u'John... |
| 973 | 7.4 | The Cider House Rules | PG-13 | Drama | 126 | [u'Tobey Maguire', u'Charlize Theron', u'Micha... |
| 976 | 7.4 | Master and Commander: The Far Side of the World | PG-13 | Action | 138 | [u'Russell Crowe', u'Paul Bettany', u'Billy Bo... |
| 978 | 7.4 | Wall Street | R | Crime | 126 | [u'Charlie Sheen', u'Michael Douglas', u'Tamar... |
538 rows × 6 columns